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COLOR REPRODUCTION SYSTEM FOR 
CARRYING OUT COLOR CORRECTION BY 
CHANGING OVER COLOR CORRECTION 
PARAMETERS ACCORDING TO IMAGES OF 
PHOTOGRAPHED SUBJECTS 

CROSS-REFERENCE TO RELATED 
APPLICATIONS 

This application is based upon and claims the benefit of 
priority from the prior Japanese Patent Applications No. 
11-151969, filed May 31, 1999; and No. 11-172772, filed 
Jun. 18, 1999, the entire contents of which are incorporated 
herein by reference. 

BACKGROUND OF THE INVENTION 

The present invention relates to a color reproduction 
system having a color image processing unit for estimating 
colors of a photographed subject from photographic signals 
obtained by photographing the subject by a color photo- 
graphing unit such as a digital camera or the like. 

As a method of estimating color information on a pho- 
tographed subject under a predetermined illumination light 
from color-image data obtained by photographing the sub- 25 
ject by a color photographing unit such as a digital camera 
or the like, there has been proposed a method of using 
foresight information on the photographed subject such as a 
base function or a correlation matrix of a spectral reflectance 
of the subject, as disclosed in Japanese Patent Application 30 
KOKAI Publication No. 11-85952. When a base function or 
a correlation matrix of a spectral reflectance of the subject is 
used, it is possible to estimate colors of the subject in high 
precision and reproduce the colors even if a photographic 
illumination light at the time of the photographing is differ- 35 
ent from an observation illumination light at the time of an 
observation. 

This conventional color reproduction system uses a spec- 
tral sensitivity of a color photographing unit, a photographic 
illumination light spectrum and an observation illumination 40 
light spectrum as information for estimating colors. How- 
ever, these data have conventionally been obtained by 
measurement using an expensive measuring instrument such 
as a spectrometer. Therefore, it has not been possible to 
provide a simple color reproduction system. 45 

On the other hand, in Jpn. Pat. Appln. KOKAI Publication 
No. 11-96333, there has been disclosed another method of 
utilizing a photographic signal of an object such as color 
charts of known spectral reflectances obtained by photo- 
graphing the object at the same time as the photographing of 50 
a subject or under the same photographing conditions. This 
method is for estimating color information of the photo- 
graphed subject under a predetermined illumination light by 
utilizing the object photographic signal, without using the 
spectral sensitivity of the color photographing unit or mea- 55 
surement data of the photographic illumination light spec- 
trum. According to this method, it is not necessary to 
measure in advance the spectral sensitivity of a color- 
photographing unit. It is not necessary to measure a photo- 
graphic illumination fight spectrum at the time of photo- 60 
graphing, cither. Therefore, this method can simplify the 
system. 

For example, it is possible to arrange as follows. At a 
hospital A, a face of a patient is photographed in colors by 
a digital camera, and at the same time, a predetermined color 65 
chart is photographed. The image data of these photographs 
are transferred to a hospital B. At the hospital B, the face 
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colors of the patient are converted into face colors of the 
patient under the observation illumination light at the hos- 
pital B, and the converted face colors are displayed on the 
monitor. When this system is employed, an expensive mea- 
5 suring instrument like a spectrometer is not necessary at the 
hospital A where the picture is taken. Thus* it becomes easy 
to obtain the data for accurate reproduction of colors. 

In the above -described color reproduction system for 
converting illumination colors by using the foresight infor- 
10 mation of a subject, it is essential that the color photograph- 
ing unit has independent spectral sensitivities by at least the 
number of base functions of spectral reflectances of the 
subject in order to make an accurate estimate of colors of the 
subject. It is known that spectral reflectances of body skin 
15 colors can be displayed in high precision by a linear sum of 
three base functions. However, in general, in order to display 
all the spectral reflectances of an object in the natural world, 
more base functions are considered to be necessary. There- 
fore, in order to carry out a simple color reproduction in high 
20 precision by using a general three-band digital camera 
having three bands of red (R), green (G), and blue (B), there 
has been a problem that a subject to be photographed is 
limited to an object that has spectral reflectances that can be 
approximated by three base functions. 

On the other hand, there has been known a color repro- 
duction system for carrying out a color reproduction by 
selecting a suitable set of an output profile and an input 
profile for one input image, as shown in FIG. 23. In FIG. 23, 
an input image from an image input unit 10 is input to a 
device -independent color converting section 11A of a color 
correcting section 11. The device-independent color con- 
verting section 11 A converts colors of the input image into 
a device-independent color image by referring to an input 
profile 111A, and outputs the device -independent color 
image to a device value converting section 11B. The device 
value converting section 11B converts the device -indepen- 
dent color image into a device value that matches the 
characteristics of an output unit by referring to an output 
profile 111B, and outputs the device value as an output 
image to an image output unit 12. 

Japanese Patent Publication No. 2678007 discloses a 
color information correcting unit for correcting a plurality of 
pieces of read information of a plurality of color components 
into corresponding recording information for a plurality of 
record open image colors based on color correction param- 
eters, in the reading of color images. 

Further, Japanese Patent Publication No. 2681181 dis- 
closes a color data correcting unit that includes an operating 
unit for correcting colors by carrying out a matrix calcula- 
tion in color data that has been obtained by digitizing an 
input signal. 

The above-described conventional color reproduction 
method in the color reproduction system is for carrying out 
a color reproduction by selecting a suitable set of an output 
profile and an input/output profile for one input image. 
According to this method, colors are reproduced based on 
the same subject characteristic data for all the objects within 
the image. In other words, colors are reproduced based on 
object spectrum statistical data. Therefore, the conversion 
precision is lowered when a plurality of objects are included 
in the image. 

According to the above-described Japanese Patent Publi- 
cation No. 2678007 and Japanese Patent Publication No. 
2681181, although a matrix is selected based on the colors 
of RGB values, only colors are corrected, and the colors are 
not converted into colors under the illumination light of the 
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observation side. Further, as subject characteristic data is not 
used, colors cannot be reproduced in high precision. 

BRIEF SUMMARY OF THE INVENTION 
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It is, therefore, an object of the present invention to 
provide a color reproduction system capable of reproducing 
colors in high precision. 

More specifically, it is an object of the present invention 
to provide a color reproduction system capable of reproduc- jq 
ing colors of all the objects included in an image in high 
precision, by dividing the image into images of a plurality of 
areas and conducting a suitable color conversion in each 
divided area. 

It is another object of the invention to provide a color 15 
reproduction system capable of reproducing colors in high 
precision, by accurately recognizing objects included in a 
photographed image and conducting color conversion suit- 
able for each object. 

It is still another object of the invention to provide a color 20 
reproduction system capable of reproducing colors in high 
precision without having characteristic information of a 
photographed subject in advance, by photographing color 
charts of known spectral reflectances and calculating char- 
acteristics of the subject from a spectral reflectance of a 25 
color chart of a value nearest a signal value obtained when 
the subject has been photographed. 

It is still another object of the invention to provide a color 
reproduction system capable of reproducing colors in high 
precision, by estimating colors of various subjects by using 30 
a color-photographing unit of a limited number of bands. 

In order to achieve the above objects, according to a first 
aspect of the present invention, there is provided a color 
reproduction system comprising: 

a color image input section; and 35 
a color correcting section for correcting colors of an 
image of a subject obtained by photographing the 
subject by the color image input section, wherein 
the color correcting section corrects colors by changing 
over a plurality of color correction parameters accord- 40 
ing to a subject image signal from the color image input 
section. 

According to a second aspect of the invention, there is 
provided a color reproduction system relating to the first 
aspect, wherein 45 
the color correcting section comprises: 
an object recognizing section for dividing a subject image 
input from the color image input section into images of 
a plurality of areas, and selecting a suitable input profile 
from among a plurality of input profiles for each 50 
divided area; 

a device-independent color converting section for con- 
verting an image of each area into a device -independent 
color image by using the input profile selected from 
among the plurality of input profiles; 55 

an image combining section for combining the device- 
independent color images converted at divided areas 
into one device -independent color image; and 

a device value image converting section for converting 
the combined device-independent color image into an 6Q 
output device value image by using a predetermined 
output profile. 

According to a third aspect of the invention, there is 
provided a color reproduction system relating to the first 
aspect, wherein the color correcting section has a plurality of 65 
input/output profiles prepared by integrating input profiles 
and output profiles together, and comprises: 



an object recognizing section for dividing an input subject 
image into images of a plurality of areas, and selecting 
a suitable input/output profile from among the plurality 
of input/output profiles for each divided area; 

a device value image converting section for converting an 
image of each area into a device value image by using 
the input/output profile selected from among the plu- 
rality of input/output profiles; and 

an image combining section for combining the device 
value images converted at divided areas into one device 
value image. 

According to a fourth aspect of the invention, there is 
provided a color reproduction system relating to the first 
aspect, further comprising: 

an object recognizing section for dividing a subject image 
data including information necessary for preparing a 
plurality of input profiles into images of a plurality of 
areas, and selecting suitable subject characteristic 
information from among a plurality of pieces of subject 
characteristic information for each divided area; 

a format converting section for converting a format based 
on selected subject characteristic information and out- 
putting image data capable of changing illumination by 
areas; 

a device-independent color converting section for calcu- 
lating an input profile from the illumination-variable 
image data, selecting a suitable input profile from 
among the plurality of input profiles based on the 
calculated input profile, and converting the image in 
each area into a device -independent color image by 
using the selected input profile; 

an image combining section for combining the device- 
independent color images converted at divided areas 
into one device-independent color image; and 

a device value image converting section for converting 
the combined device-independent color image into an 
output device value image by using a predetermined 
output profile. 

According to a fifth aspect of the invention, there is 
provided a color reproduction system relating to the first 
aspect, further comprising: 

an area dividing section for dividing an input image into 
images of a plurality of areas; 

a device -independent color converting section for con- 
verting the image in each divided area into a device- 
independent color image by using the plurality of input 
profiles in parallel; 

a conversion result selecting section for selecting a 
device-independent color image for each area by com- 
paring the plurality of converted device-independent 
color images; 

an image combining section for converting the device- 
independent color images selected at divided areas into 
one device-independent color image; and 

a device value image converting section for converting 
the combined device-independent color image into an 
output device value image by using a predetermined 
output profile. 

According to a sixth aspect of the invention, there is 
provided a color reproduction system relating to the first 
aspect, further comprising: 

an area dividing section for dividing an input image into 

images of a plurality of areas; 
a first device-independent color converting section for 
converting the image in each divided area into a 
device-independent color image by using one input 
profile; 
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a second device -independent color converting section for 
converting the input image in each divided area into a 
device-independent color image by using an input 
profile selected from among a plurality of input profiles 
for each area based on a result of the conversion in the 5 
first device -independent color converting section; 

an image combining section for converting the device- 
independent color images converted at divided areas 
into one device-independent color image; and 

a device value image converting section for converting 10 
the combined device-independent color image into an 
output device value image by using a predetermined 
output profile. 

According to a seventh aspect of the invention, there is 
provided a color reproduction system relating to the first 15 
aspect, further comprising: 

an area dividing section for dividing an input image into 

images of a plurality of areas; 
a first device-independent color converting section for 
converting the image in each divided area into a 20 
device -independent color image by using one input 
profile; 

a spectral reflectance database for storing information on 
spectral reflectances of a plurality of objects; 

a conversion result judging section forjudging a result of 25 
the conversion at the first device-independent color 
converting section by referring to the spectral reflec- 
tance database; 

a subject characteristic information calculating section for 
calculating subject characteristic information by select- 30 
ing information in the spectral reflectance database 
based on a result of the decision by the conversion 
result judging section; 

a second device -independent color converting section for 
converting the input image in each divided area into a 
device-independent color image by using an input 
profile selected from among a plurality of input profiles 
prepared based on the subject characteristic informa- 
tion calculated by the subject characteristic information 
calculating section; 

an image combining section for converting the device- 
independent color images converted at divided areas 
into one device -independent color image, and 

a device value image converting section for converting ^ 
the combined device-independent color image into an 
output device value image by using a predetermined 
output profile. 

According to an eighth aspect of the invention, there is 
provided a color reproduction system relating to the first 5Q 
aspect, further comprising: 

an object recognizing section for selecting an input profile 
to be used from among a plurality of input profiles 
based on a signal value of an image obtained by 
photographing using a plurality of color charts of 55 
known spectral reflectances; 

a device-independent color converting section for con- 
verting an image of each area into a device-independent 
color image by using the input profile selected from 
among the plurality of input profiles; 6G 

an image combining section for combining the device - 
independent color images converted at divided areas 
into one device-independent color image; and 

a device value image converting section for converting 
the combined device-independent color image into an 65 
output device value image by using a predetermined 
output profile. 
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According to a ninth aspect of the invention, there is 
provided a color reproduction system relating to the first 
aspect, further comprising: 

a spectral reflectance database having spectral reflec- 
tances of color charts; 

a subject characteristic calculating section for calculating 
subject characteristic information from a color-chart 
photographic image based on signal values of an image 
obtained by photographing using a plurality of color 
charts of known spectral reflectances and data of the 
spectral reflectance database; 

an object recognizing section for dividing a subject pho- 
tographic image into images of a plurality of areas, and 
selecting an input profile to be used from among a 
plurality of input profiles based on the calculated 
subject characteristic information; 

a device -independent color converting section for con- 
verting an image of each area into a device-independent 
color image by using the input profile selected from 
among the plurality of input profiles; 

an image combining section for combining the device- 
independent color images converted at divided areas 
into one device-independent color image; and 

a device value image converting section for converting 
the combined device-independent color image into an 
output device value image by using a predetermined 
output profile. 

According to a tenth aspect of the invention, there is 
provided a color reproduction system relating to the second 
aspect, wherein 

the input profile is prepared based on at least one of image 
input unit information that includes photographic char- 
acteristics of an image input unit used for the photo- 
graphing and information on a set state of the color 
image input section, observation illumination light 
information, and subject characteristic information that 
represents statistical characteristics of spectral reflec- 
tances of the photographed subject. 

According to an eleventh aspect of the invention, there is 
provided a color reproduction system relating to the first 
aspect, wherein 

the color correcting section comprises a color estimating 
section for estimating tristimulus values of the subject 
under an observation illumination fight, based on sub- 
ject photographic signals obtained by photographing 
the subject by the color image input section, spectral 
sensitivity of the color image input section, color 
matching functions, a spectrum of photographic illu- 
mination light, a spectrum of observation illumination 
light and statistical data of spectral reflectances of the 
subject, and 

the color estimating section uses the statistical data which 
is changed over according to the subject photographic 
signals. 

According to a twelfth aspect of the invention, there is 
provided a color reproduction system relating to the first 
aspect, wherein 

the color correcting section comprises a color estimating 
section for estimating tristimulus values of the subject 
under an observation illumination light, based on object 
photographic signals obtained by photographing a plu- 
rality of objects of known spectral reflectances by the 
color image input section, subject photographic signals 
obtained by photographing the subject by the color 
image input section, color matching functions, a spec- 
trum of observation illumination fight and statistical 
data of spectral reflectances of the subject, and 
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the color estimating section uses the statistical data which 
is changed over according to the subject photographic 
signals. 

According to a 13th aspect of the invention, there is 
provided a color reproduction system relating to the eleventh 5 
aspect, wherein the color estimating section uses the statis- 
tical data which is changed over based on a relationship 
between spectral reflectance data for calculating statistics of 
spectral reflectances and the subject photographic signals. 

According to a 14th aspect of the invention, there is 10 
provided a color reproduction system relating to the twelfth 
aspect, wherein the color estimating section uses the statis- 
tical data which is changed over based on a relationship 
between spectral reflectance data for calculating statistics of 
spectral reflectances and the subject photographic signals. 15 

According to a 15th aspect of the invention, there is 
provided a color reproduction system relating to the twelfth 
aspect, wherein the color estimating section changes over 
the statistical data based on a relationship between spectral 
reflectance data for calculating statistics of spectral reflec- 20 
tances of the subject obtained from a relationship between 
the object photographic signals and spectral reflectances of 
the objects, and the subject photographic signals. 

Additional objects and advantages of the invention will be 
set forth in the description which follows, and in part will be 25 
obvious from the description, or may be learned by practice 
of the invention. The objects and advantages of the invention 
may be realized and obtained by means of the instrumen- 
talities and combinations particularly pointed out hereinaf- 
ter. 30 

BRIEF DESCRIPTION OF THE SEVERAL 
VIEWS OF IHE DRAWING 

The accompanying drawings, which are incorporated in 35 
and constitute a part of the specification, illustrate presently 
preferred embodiments of the invention, and together with 
the general description given above and the detailed descrip- 
tion of the preferred embodiments given below, serve to 
explain the principles of the invention. 40 

FIG. 1 is a diagram showing a structure of a color 
reproduction system relating to a first embodiment of the 
present invention. 

FIG. 2 is a diagram showing a detailed structure of the 
device -independent color converting section 202. 45 

FIG. 3 is a diagram showing a detailed structure of a 
device value converting section 204. 

FIG. 4 is a diagram showing a structure of a modification 
of the first embodiment. 

FIG. 5 is a diagram showing a distribution of a spectral 50 
reflectance space of subjects in a multi-dimensional space of 
a spectrum. 

FIG. 6 is a diagram showing a structure of a color 
reproduction system relating to a second embodiment of the 
present invention. 55 

FIG. 7 is a diagram showing a detailed structure of a 
device value converting section 222. 

FIG. 8 is a diagram showing a structure of a modification 
of the second embodiment. 

FIG. 9 is a diagram showing a structure of a color 60 
reproduction system relating to a third embodiment of the 
present invention. 

FIG. 10 is a diagram showing a format of illumination 
variable image data by areas 244. 

FIG. 11 is a diagram showing a structure of a color 65 
reproduction system relating to a fourth embodiment of the 
present invention. 



FIG. 12 is a diagram showing distributions of certain 
specific color spaces in a color space of various objects in a 
natural world. 

FIG. 13 is a flowchart showing details of an algorithm for 
carrying out a processing of the fourth embodiment. 

FIG. 14 is a diagram showing a structure of a color 
reproduction system relating to a fifth embodiment of the 
present invention. 

FIG. 15 is a flowchart showing details of an algorithm for 
carrying out a processing of the fifth embodiment. 

FIG. 16 is a diagram showing a structure of a color 
reproduction system relating to a sixth embodiment of the 
present invention. 

FIG. 17 is a diagram for explaining a seventh embodiment 
of the present invention. 

FIG. 18 is a diagram showing a structure of a color 
reproduction system relating to the seventh embodiment of 
the present invention. 

FIG. 19 is a diagram showing a distribution of a spectral 
reflectance space of a subject in a multi-dimensional space 
of a spectrum. 

FIG. 20 is a diagram for explaining an eighth embodiment 
of the present invention. 

FIG. 21 is a diagram showing a structure of a color 
reproduction system relating to the eighth embodiment of 
the present invention. 

FIG. 22 is a diagram for explaining a ninth embodiment 
of the present invention. 

FIG. 23 is a diagram showing a structure of a conven- 
tional color reproduction system. 

FIG. 24 is a block diagram showing a structure of a color 
reproduction system relating to a tenth embodiment of the 
present invention. 

FIG. 25 is a diagram showing a detailed example of a 
subject spectral reflectance database in the tenth embodi- 
ment. 

FIG. 26 is a block diagram showing a structure of an LUT 
data calculating unit in the tenth embodiment. 

FIG. 27 is a diagram showing a detailed example of a 
database photographic signal in the tenth embodiment. 

FIG. 28 is a conceptual diagram for explaining sample 
photographic signals and a selection of a nearest photo- 
graphic signal in the tenth embodiment. 

FIG. 29 is a block diagram showing a structure of a color 
conversion LUT calculating unit in FIG. 26. 

FIG. 30 is a block diagram showing a structure of a 
monitor input signal calculating unit in FIG. 29. 

FIG. 31 is a block diagram showing a structure of a data 
interpolating unit in FIG. 29. 

FIG. 32 is a block diagram showing a structure of a color 
reproduction system relating to an eleventh embodiment of 
the present invention. 

FIG. 33 is a block diagram showing a structure of an LUT 
data calculating unit in the eleventh embodiment. 

FIG. 34 is a block diagram showing a structure of a color 
conversion LUT calculating unit in FIG. 33. 

DETAILED DESCRIPTION OF THE 
INVENTION 

Embodiments of the present invention will be now be 
explained in detail with reference to the attached drawings. 



First Embodiment 

FIG. 1 is a diagram showing a structure of a color 
reproduction system relating to a first embodiment of the 
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present invenLion. This color reproduction system consists using any illumination light such as, for example, a rluo- 

of an image input unit 100 for photographing an image of an rescent light, an incandescent light, solar beams, etc. 

object, a color correcting section 200 for correcting the input Further,, the use of the observation illumination light 

image and generating an output image, and an image output information makes it possible to convert the colors of a 

unit 300. 5 photographed object into colors under any illumination 

In the color correcting section 200, an object recognizing %ht. For example, by a remote-control reproduction of 

section 201 recognizes an object within an input image. In colors of an object under a different illumination, it is 

other words, an area dividing section 201-1 of the object possible to display the colors of the object as if the object is 

recognizing section 201 divides the input image into images positioned at the place of observation. 

of a plurality of areas. An input profile selecting section 1° Further, the use of the subject characteristic information 

201-2 selects a suitable input profile from among a plurality makes it possible to estimate a color reproduction image in 

of input profiles for each divided area by changing over a hi g h precision even if the input image has little spectrum 

switch 205. A device -independent color converting section information. In other words, it is possible to estimate a 

202 has a memory unit for storing a plurality of input spectral reflectance of an object in high precision when an 

profiles 1 to n (2021 to 202«). The device-independent color * 5 image input unit is a digital camera for photographing a 

converting section 202 converts an image in each area into color image of RGB three bands instead of a digital camera 

a device-independent color image by using an input profile for photographing a color image of a plurality of spectra, 

that has been selected from among the plurality of input ih e subject characteristic information may be a correlation 

profiles 1 to n (2021 to 202n) by the input profile selecting matrix or a base function that represents statistical charac- 

section 201-2. An image combining section 203 combines 20 teristics of a spectral reflectance. 

the device-independent color images converted into images Further, the use of the image output unit information 
of divided areas into one device-independent color image. A makes it possible to convert in high precision a device- 
device value converting section 204 converts the combined independent color image into an output image that matches 
device-independent color image into an output image by the image output unit 300. This makes it possible to repro- 
referring to an output profile 2040. 25 duce colors by taking into consideration differences in 
FIG. 2 is a diagram showing a detailed structure of the characteristics and variations between monitors and between 
device -independent color converting section 202. The printers. 

device-independent color converting section 202 consists of explained above according to the present cmbodi- 

the above memory section for storing the plurality of input mcnt ' an ! n P ul ,ma S e * d iv,ded * ™ages of a plurality of 

profiles 1 to n (2021 to 202«), an input profile preparing 30 areas and an input profile is to be used for each d.vided area 

section 2031 for calculating an input profile based on at least 1S sclected from among a plurality of input profiles. The 

one set of information of characteristics and setting of an m P ut page 's converted into a device-independent color 

image input unit used for a photographing (image input unit "»*, b * usln S a sele ^ lcd m P ut P rofile 11 f P os ?! ble 

information), spectrum data of an illumination light used for to . make an accurate , decls,on on the Photographed object 

photographing an image of an object by the image input unit 35 ™ tb f an image and to use an input profile that is suitable 

(photographic illumination light information), spectrum data for ,h u e , subject characteristics of the object. Therefore, it is 

of an illumination light at a position where the image of the P<f slble «o achieve a color reproduction ,n h,gh precision for 

photographed object is to be observed (observation illumi- aU pa ! LS / , thC magC ,- . r , . , . 

nation light information), and statistical characteristics of a U FIG 4 shows a modification of the l.rst embodiment. This 

spectral reflectance of the photographed object (subject 40 show f. an f am P le of an "™B e Pressing unit having a 

characteristic information), and an input profile working Polity of input profiles (1 to N) by using the subject 

section 2030 for working the prepared input profile onto the characteristic information. Usually, it is difficult to pick up 

input image in each divided area to convert the input image an ™age of only a desired object in one picture. A back- 

into a device-independent color image. ground of the selected object and other objects are also 

„ „ . ,. , . , ^ r . i • , 45 included in the same picture. When colors of these objects 

FIG. 3 is diagram showing details of the device value afe ^ for repro d U ction by using only the input 

converting section 204. The device value converting section filg ^ ^ cnaraclerist ic information of one subject, it 

204 consists of an output profile ^prep anng section 2043 for ^ ibJe tQ roduce the colors of the desired object in 

calculating an output profile 2042 based on input interna- high precision. However, colors of the background and other 

tion on an output unit image output unit information), and M ^ afe QOt rcproduced U1 hi ^ h predsion . 

an output profile working section 2041 for working the Tq solve ^ blem? a color correcting ^ ction 20 0 in 

prepared output profile 2042 onto a device -independent nG 4 uses a luraHt of { { files 2(m to 202N mat 

color image to convert the device-independent color image URe characteristic informiltl - on of various subject s including 

into an output image. a pcrson? aa anima i and a p i anL A n object recognizing 

The use of the above image input unit information makes 55 sec tion 201 decides what has been photographed in a certain 
it possible to convert an input image into a device-indepen- area within the image based on a signal value of an input 
dent color image by taking into consideration differences in image, and selects a suitable input profile from among a 
characteristics of various image input units and di fferences plurality of input profiles. A device -independent color con- 
in the setting of photographing. It is possible to achieve a verting section 202 works an input profile selected from 
color reproduction in high precision when the image input 6D araon g a plurality of input profiles 2021 to 202N onto the 
unit 100 is a multi-spectrum camera or a digital camera for input image, thereby to prepare a device-independent color 
photographing a plurality of spectrum images. image. It is also possible to prepare a plurality of pieces of 

Further, the use of the photographic illumination light subject characteristic information in advance based on col- 
information makes it possible to cancel influence of the ors of objects or based on substances that constitute objects, 
illumination light at the time of the photographing. In other 65 Assume that a spectral reflectance space of subjects is 
words, it is possible to calculate an accurate spectral reflec- distributed in a multi -dimensional space of a spectrum as 
tance of an object itself when the object is photographed by shown in FIG. 5. In FIG. 5, h (X) represents a spectral 
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sensitivity of a camera. On the spectral sensitivity of h (K) f 
a subject spectral reflectance that takes a value of a signal 
value pi represents all the points on a line perpendicular to 
h (X). When subject information of only one subject is used 
like the conventional method, p2 of a subject A and p3 of a 5 
subject B take the same signal value. Therefore, in the 
estimate of spectral reflectances of these subjects based on 
this signal value, these subjects take the same spectral 
reflectance. 

When two sets of subject characteristic information are 10 
used for the subject A and the subject B like the present 
embodiment, it becomes possible to estimate two spectral 
reflectances for the p2 and p3, based on the same signal 
value pi. As explained above, by using a plurality of sets of 
subject characteristics, it is possible to estimate spectral 15 
reflectances of subjects in high precision. 

Second Embodiment 

FIG. 6 is a diagram showing a structure of a color 20 
reproduction system relating to a second embodiment of the 
present invention. This color reproduction system consists 
of an image input unit 100 for photographing an image of an 
object, a color correcting section 220 for correcting the input 
image and generating an output image, and an image output 25 
unit 300. 

In the color correcting section 220, an object recognizing 
section 221 recognizes an object within an input image. In 
other words, an area dividing section 221-1 of the object 
recognizing section 221 divides the input image into images 30 
of a plurality of areas. An input/output profile selecting 
section 221-2 selects a suitable input/output profile from 
among a plurality of input/output profiles for each divided 
area by changing over a switch 224. A device value con- 
verting section 222 has a memory unit for storing a plurality 35 
of input/output profiles 1 to N (2221 to 222N). The device 
value converting section 222 converts an image in each area 
into a device-independent color image by using an input/ 
output profile that has been selected from among the plu- 
rality of input/output profiles 1 to N (2221 to 222N) by the 40 
input/output profile selecting section 221-2. An image com- 
bining section 223 combines the device-independent color 
images converted into images of divided areas into one 
device-independent color image. 

FIG. 7 is a diagram showing a detailed structure of the 45 
device value converting section 222. The device value 
converting section 222 consists of an input profile 2281, 
2282, 228N preparing section 2270 for calculating an input 
profile based on at least one set of information of charac- 
teristics and setting of an image input unit used for a 50 
photographing (image input unit information), spectrum 
data of an illumination light used for photographing an 
image of an object by the image input unit 100 (photo- 
graphic illumination light information), spectrum data of an 
illumination light at a position where the image of the 55 
photographed object is to be observed (observation illumi- 
nation light information), and statistical characteristics of a 
spectral reflectance of the photographed object (subject 
characteristic information), an output profile preparing sec- 
tion 2260 for calculating an output profile 2250 based on 60 
input information on an image output unit (image output unit 
information), an input/output profile preparing section 2240 
for combining the prepared input profile and the output 
profile to prepare an input/output profile, and an input/output 
profile working section 2230 for working the prepared 65 
input/output profile onto the input image to convert the input 
image into an output image. 
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The use of an input/output profile prepared by combining 
the input profile and the output profile of the first embodi- 
ment makes it possible to convert the input image into a 
color-reproduced output image at a higher speed. 

FIG. 8 shows a modification of the second embodiment. 
Usually, it is difficult to pick up an image of only a desired 
object in one picture. A background of the selected object 
and other objects are also included in the same picture. 
When colors of these objects are processed for reproduction 
by using only the input profile that uses characteristic 
information of one subject like the conventional method, it 
is possible to reproduce the colors of the desired object in 
high precision. However, colors of the background and other 
objects are not reproduced in high precision. 

To solve this problem, a color correcting section 220 in 
FIG. 8 uses a plurality of input/output profiles 2221 to 222N 
that use characteristic information of various subjects 
including a person, an animal and a plant. An object recog- 
nizing section 221 decides what has been photographed in a 
certain area within the image based on a signal value of an 
input image, and selects a suitable input/output profile from 
among a plurality of input/output profiles. A input/output 
profile working section 222 works an input/output profile 
selected from among a plurality of input/output profiles 
2221 to 222N onto the input image, thereby to prepare a 
device -independent color image. 

Third Embodiment 

FIG. 9 is a diagram showing a structure of a color 
reproduction system relating to a third embodiment of the 
present invention. This color reproduction system consists 
of an image input unit 100 for photographing an image of an 
object, a color correction pre-processing section 240 for 
converting an input image into illumination variable image 
data (illumination variable image data by areas) 244, a color 
correcting section 245 for preparing an input profile from the 
illumination variable image data, converting the input image 
into a device-independent color image, and then converting 
the device -in dependent color image into an output image 
that matches the image output unit, and the image output 
unit 300 for oulputting (displaying and printing) an output 
image. 

The third embodiment is characterized in that the input 
image has information necessary for preparing a plurality of 
input profiles, and has an illumination-variable data struc- 
ture that uses an input profile according to each area within 
the image. In other words, the input image has the image 
data of a subject and any one set of the image input unit 
information, photographic illumination light information, 
observation illumination light information, and characteris- 
tic information of a plurality of subjects, that is necessary for 
preparing each input profile. 

In the color correction pre-processing section 240, an area 
dividing section 241-1 of an object recognizing section 241 
divides an input image into images of divided areas. A 
subject characteristic selecting section 241-2 selects a spe- 
cific subject characteristic from among a plurality of subject 
characteristics included in the input image. A format con- 
verting section 242 carries out a format conversion based on 
a result of a selection by a photographic illumination light 
spectrum selecting section 243, and outputs a converted data 
as the illumination variable image data by areas 244. 

In the color correcting section 245, an input profile 
calculating section 251 calculates an input profile from the 
illumination variable image data by areas. A device-inde- 
pendent color converting section 246 converts the Alumina- 
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tion variable image data by areas 244 into a device -inde- 
pendent color image by using an input profile that has been 
selected from among a plurality of input profiles 1 to N 
(2461 to 246N) corresponding to the calculated input profile 
by changing over a switch 250. An observation illumination 5 
light spectrum data selecting section 247 is also provided, as 
shown. An image combining section 248 combines devicc- 
iiidependent color images by areas into one device -indepen- 
dent color image. A device value converting section 249 
converts the device-independent color image into an output id 
image that matches the image output unit by using an output 
profile 2490. 

FIG. 10 is a diagram showing a format of the illumination 
variable image data by areas 244. The illumination variable 
image data by areas 244 consists of a header 244-1 that is 
includes information on a marker, a format version, a size of 
an image, number of dimensions of the image, number of 
areas of the image, etc., photographic characteristic infor- 
mation 244-2 that includes information on camera spectral 
sensitivity, photographing shutter speed, photographing dia- 20 
phragm, etc., photographic illumination light information 
244-3 that includes information on a starting wavelength, an 
ending wavelength, a wavelength interval, spectrum data, 
etc., an area table 244-4 that stores information on each area 
to which each pixel belongs (object spectrum statistical data 25 
to be used), subject characteristic information 1 (244-5) that 
includes information on an object base function and eigen 
value data 1, subject characteristic information 2 (244-6) 
that includes information on an object base function and 
eigen value data 2, subject characteristic information 3 30 
(244-7) that includes information on an object base function 
and eigen value data 3, and image data 244-8 that includes 
data on monochromatic imagexnumber of dimensions of the 
image. Although FIG. 10 shows data having the three sets of 
subject characteristic information, it is also possible to 35 
include more sets of subject characteristic information. 

Fourth Embodiment 

FIG. 11 is a diagram showing a structure of a color 40 
reproduction system relating to a fourth embodiment of the 
present invention. The color reproduction system consists of 
an image input unit 100 for photographing an image of an 
object, a color correcting section 260, and an image output 
unit 300 for outputting (displaying and printing) an output 45 
image. 

The fourth embodiment is characterized in that after an 
input image has been divided into images of a plurality of 
areas, the images in these areas are converted into device- 
independent color images in parallel by using a plurality of 50 
input profiles. 

In the color correcting section 260, an area dividing 
section 261 divides an input image into images of a plurality 
of areas. A device-independent color converting section 262 
converts image data for each area into a device-independent 55 
color image by using a plurality of input profiles 1 to 3 (2621 
to 2623). A conversion result selecting section 263 compares 
between results 1, 2 and 3 of device-independent color 
images obtained by the conversion based on the plurality of 
input profiles, and selects one device -independent color 60 
image. An image combining section 264 combines device- 
independent color images that have been selected by areas 
into one device-independent color image. 

A device value converting section 265 converts the 
device -independent color image into an output image that 65 
matches the image output unit by using an output profile 
2650. 



Colors of a human body skin and a cloth of clothes form 
a limited small space in a total color space of various objects 
in the natural world, as shown in FIG, 12. Therefore, when 
colors of a specific subject arc reproduced based on subject 
characteristic information that represents total objects in the 
natural world, there is small error in the reproduced colors. 
However, on the contrary, when colors of other objects are 
reproduced based on characteristic information of a certain 
specific subject, there arises a large error in the reproduced 
colors. 

FIG. 13 is a flowchart showing details of an algorithm for 
carrying out a processing of the fourth embodiment. A 
threshold value used in the following explanation is one 
example, and the threshold value is not limited to this 
example. A color difference 3.2 means a color difference in 
L*a*b* space. This is a difference between colors (color 
difference) that cannot be distinctly recognized as a different 
color by human beings when the object is in a distance. 
Although the three sets of subject characteristic information 
are used for processing them in parallel in the present 
embodiment, it is also possible to use more sets of subject 
characteristic information. 

First, a chromaticity value is calculated based on total 
subject characteristic information (at step SI). Next, a 
chromaticity value is calculated based on specific subject 
characteristic information 1 (at step S2). Next, a chromatic- 
ity value is calculated based on specific subject character- 
istic information 2 (at step S3). A decision is made as to 
whether a color difference between the chromaticity value 
obtained at the step Si and the chromaticity value obtained 
at the step S2 is within a threshold value (A 1:3 2) or not (at 
step S4). When a decision is YES, a result of the specific 
subject characteristic information 1 is used (at step S8), and 
images are combined together (at step S9). 

When a decision made at the step S4 is NO, a decision is 
made as to whether a color difference between the chroma- 
ticity value obtained at the step vSl and the chromaticity 
value obtained at the step S3 is within the threshold value (A 
E3.2) or not (at step S5). When a decision is YES, a result 
of the specific subject characteristic information 2 is used (at 
step S6), and images are combined together (at step S9). 
When a decision made at the step S5 is NO, a result of the 
total subject characteristic information is used (at step S7), 
and images are combined together (at step S9). 

Fifth Embodiment 

FIG. 14 is a diagram showing a structure of a color 
reproduction system relating to a fifth embodiment of the 
present invention. This color reproduction system consists 
of an image input unit 100 for photographing an image of an 
object, a color correcting section 280 for correcting the input 
image and generating an output image, and an image output 
unit 300 for outputting (displaying and printing) an output 
image. 

The fifth embodiment is characterized in that after an 
input image has been divided into images of a plurality of 
areas, the images in these areas are converted into device- 
independent color images in hierarchy by using a plurality of 
input profiles. 

In the color correcting section 280, an area dividing 
section 281 divides an input image into images of a plurality 
of areas. A device-independent color converting section 1 
(282) converts image data for each area into a device- 
independent color image by first using one input profile 1 
(2821). A conversion result judging section 283 decides one 
input profile to be used from among the plurality of input 
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profiles based on a result of the conversion by the device- image and generating an output image, and an image output 
independent color converting section 1 (282). A device- unit 300 for outputting (displaying and printing) an output 
independent color converting section 2 (284) converts the image. 

input image divided into areas into device-independent color ^ present embodirnent is characterized in that the color 

images again by using an irxput profile selected from input 5 duclion tem has a lral refiectance database, and 

profiles 2 and 3 (2841 and 2842) by changing over a switch ^ characteristic mformation for cach input 

287 according to a result of a decision made by the conver- . , . , - , , . . „ , . 

. ? . . „„,, . ... ,. image by calculation based on spectral reflectance data, 
sion result judging section 2S3. An image combining section 

285 combines the device-independent color images by areas In ,he correcting section 310, an area dividing 

into one device-independent color image. 10 section 311 divides an input image into images of a plurality 

A device value converting section 286 converts the of areas. A device-independent color converting section 1 
device-independent color image into an output image that (312) as a spectral reflectance estimating section first con- 
matches the image output unit by using an output profile verts image data for each area into a device-independent 
2860. color image of an estimated spectral reflectance by referring 

FIG. 15 is a flowchart showing details of an algorithm for is to one input profile 1 (3121) as a base of all natural objects, 
carrying out a processing of the fifth embodiment. First, a a conversion result judging section 313 compares a 
chromaticity value is calculated based on total subject spectral reflectance database 317 in which spectral reflec- 
characteristic information of natural objects (at step S20). A tances of various objects have been registered with the 
decision is made as to whether the chromaticity value spectral reflectance stored in the spectral reflectance data- 
obtained at the step S20 is sufficiently close to the chroma- 2(1 base 317 based on the result of the device-independent color 
ticity value of a specific subject 1 (for example, colors of a converting section 1 (312), and checks chromaticity data 
human body skin) (at step S21). When a decision is YES, a that is closest to estimated data. 

chromaticity value is calculated based on the specific subject A subject characteristic information calculating section 

characteristic information 1 (at step S25). Next, a result of 31fi data fa ^ fal refleclance database 317 

the specific subject characteristic information 1 is used (at 25 based M ^ resu „ „ f ^ conversion resu]t ■ d i section 

step S26), and images are combined together (at step S24). 313 calculates subject characteristic data/and inputs the 

When a decision made at the step S21 is NO, a decision ca]culated data t0 a device-independent color converting 

is made as to whether the chromaticity value is sufficiently 2 (31S) A device . indcperjdcr) t color converting 

±5f l ° i e chromaticit y valu *£ a s P ec u lflc sub J ect 2 <f ste .P section 2 (315) as a spectral reflectance estimating section 2 

522) When a decision ,s YES, a chromaticity value is 3D s aQ ; fi j £ % (a ba£e rf a fed . color Qatural 

calculated based on the specific subject characteristic infor- qW } 3152 an ■ flfe 3 (> bas£ of a hlue ^ olo[ 

mation 2 (at step S27). Next, a result of the specific subject natural object) 3153 and an ; file 4 (a base of a 

characteristic information 2 is used (at step S28), and images ^^.^ natura l ob ject) 3154, based on calculated subject 

are combined together (at step S24) characterLstic data, and stores these files in a memory 

When a decision made at the step S22 is NO, a result of 35 dcvic e-independent color converting section 2 

the total subject characteristic information is used (at step (315) converts thc j t ^ djvided ^ are as into a 

523) , and images are combined together (at step S24). device-independent color image 1 again by using a input 
In other words, in the present embodiment, a chromaticity fi)e from a the luraKt of ; , ofi j es 2 

value is calculated based on the total subject characteristic tQ 4 (3152 M 3154) b ch a i ovcr , sw ; tch 324 accor ding 

information of natural objects, llie conversion result judg- 40 [o a dcdsjon made b , he conversk) n rcsu l t judging section 

ing section 283 checks whether the chromaticity value m ^ jf . a a , rej1ectance estirnate< J by the input 

obtained is sufficiently close to the chromaticity value 01 the profile 2 3 or 4 

specific subject 1 (for example, colors of a human body skin) ^ ~ 
or the chromaticity value is sufficiently close to the chro- J *£ e J? m 1Q P Ut P r °^ eS 1 , (312 u 1) ' 2 (3152 > 3 (3153) 
maticity value of a specific subject 2 (for example, colors of 45 4 ( 315 J 4) .f, re Pupated based on thc setting of a photograph- 
cloth). When thc chromaticity is sufficiently clo.se to a mg side lUumination hghl setting section 314 and an image 
certain specific subject, an area image is converted into a ln P ul umt characteristic setting section 318 and subject 

a^,:^ :*a, ~ • ... , • ■ . characteristic information calculated from data withm a 

device-independent color image again by using an input 

profile calculated from the specific subject characteristic spectral rellcctance D13. 

information according to this specific subject. In other cases, 50 A device-independent color converting section 3 (319) as 
a result of a conversion using an input profile calculated arj illumination converting section converts the device- 
based on the Lotal subject characteristic information of independent color image 1 into a device-independent color 
natural objects is used. image 2 as a chromaticity value obtained by illumination 
In the present embodiment, there has been explained a conversion based on an input profile 5 (illumination con- 
case of carrying out image color conversion by using one set 55 version) 3195, by referring to the input profile 5. Thc input 
of total subject information and two sets of specific subject profile 5 is prepared based on input profile calculation data 
information. However, it is also possible to use more sets of from an observation side illumination light setting section 
subject information or to carry out a hierarchical conversion 320. 

by dividing the process into more steps. An image combining section 322 combines input device- 

60 independent color images 2 by areas. A device value con- 
Sixth Embodiment verting section 323 converts thc combined image into an 

output image that matches the device by referring to an 
FIG. 16 is a diagram showing a structure of a color output profile 3230. The output profile 3230 is prepared 
reproduction system relating to a sixth embodiment of thc based on output profile calculation data that is set from an 
present invention. This color reproduction system consists 65 image output unit characteristic setting section 321. The 
of an image input unit 100 for photographing an image of an output image obtained by thc device value converting sec- 
object, a color correcting section 310 for correcting the input tion 323 is output to the image output unit 300. 
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The subject characteristic information may be either a 
correlation matrix or a base function that represents statis- 
tical characteristics of a spectral reflectance. 

Seventh Embodiment 

FIG. 17 is a diagram for explaining a seventh embodiment 
of the present invention. According to the seventh embodi- 
ment, a digital camera 406 photographs a color chart 400 of 
a known spectral reflectance to form a color chart photo- 
graphic image 401. 'ITie color chart photographic image 401 
is input to an input profile selecting section 402. Based on 
a signal value of this image, an input profile 403 to be used 
for an illumination conversion of an object photographic 
image is selected and is input to an input profile working 
section 404. At the same time, an object photographic image 
407 of an object 405 photographed by the digital camera 406 
is input to the input profile working section 404. The input 
profile working section 404 works the input profile 403 onto 
the object photographic image 407 to convert the object 
photographic image into a device -independent color image. 

FIG. 18 is a diagram showing a structure of a color 
reproduction system relating to the seventh embodiment of 
the present invention. This color reproduction system con- 
sists of an image input unit 100 for photographing an image 
of an object and an image of a color chart, a color correcting 
section 330, and an image output unit 300 for outputting 
(displaying and printing) an output image. 

In the color correcting section 330, an image type select- 
ing section 331 selects a type of an image. An object 
recognizing section 332 divides an object photographic 
image into images of a plurality of areas, and selects an input 
profile to be used from among a plurality of input profiles by 
referring to signal values of the color chart photographic 
image and the object photographic image. An input profile 
preparing section 336 in a device -independent color con- 
verting section 333 prepares a plurality of input profiles 
3351 to 335N to be selected by the object recognizing 
section 332. An input profile working section 334 prepares 
a device-independent color image by working an input 
profile that has bee selected from among the plurality of 
input profiles 1 to N (3351 to 335N) by changing over a 
switch 332-1 onto the object photographic image. An image 
combining section 337 combines device-independent color 
images by areas into one device-independent color image. A 
device value converting section 338 converts the device- 
independent color image into an output image that matches 
the device by referring to an output profile 3380. 

Assume that a spectral reflectance space of subjects is 
distributed in a multi-dimensional space of a spectrum as 
shown in FIG. 19. In this case, h (k) represents a spectral 
sensitivity of a camera. On the spectral sensitivity of h (X), 
a subject spectral reflectance that takes a value of a certain 
signal value represents all the points on a line perpendicular 
to h (X). 

When one conversion matrix is used like the conventional 
method, the spectral reflectance of a subject is projected to 
one axis of poor linearity, and thereafter the spectral reflec- 
tance is projected to the spectral sensitivity h (A.) of the 
camera. According to the present embodiment, the conver- 
sion matrix to be used is changed over depending on the 
signal value. Therefore, it is possible lo project the spectral 
reflectance to a plurality of axes of satisfactory linearity. As 
a result, it is possible to obtain color reproduction in high 
precision. 
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Eighth Embodiment 

FIG. 20 is a diagram for explaining an eighth embodiment 
of the present invention. According to the eighth embodi- 

5 nient, a digital camera 508 photographs a color chart 500 of 
a known spectral reflectance to form a color chart photo- 
graphic image 501. The color chart photographic image 501 
is input to a subject characteristic information calculating 
section 503. The subject characteristic information calculat- 

10 ing section 503 calculates subject characteristic information 
based on data in a color chart spectral reflectance database 
(DB) 502. An input profile preparing section 504 prepares an 
input profile 505 for illumination-converting a subject pho- 
tographic image 509 based on the subject characteristic 

15 information calculated. The subject photographic image 509 
is obtained by photographing an object 506 by the digital 
camera 508. An input profile working section 507 works the 
input profile 505 onto the subject photographic image 509 to 
obtain and output a device- independent color image. 

20 FIG. 21 is a diagram showing a structure of a color 
reproduction system relating to the eighth embodiment of 
the present invention. This color reproduction system con- 
sists of an image input unit 100 for photographing an image 
of an object, a color correcting section 430, and an image 

25 output unit 300 for outputting (displaying and printing) an 
output image. 

In the color correcting section 430, an image type select- 
ing section 431 selects a type of an image. An object 
recognizing section 432 divides an object photographic 

30 image into im ages of a plurality of areas, and selects an input 
profile lo be used from among a plurality of input profiles 
based on the object photographic image divided into areas. 
An input profile preparing section 436 in a device-indepen- 
dent color converting section 433 prepares a plurality of 

35 input profiles 1 to N (4351 to 435N) to be selected by the 
object recognizing section 432. In the present embodiment, 
the color correcting section 430 includes a subject charac- 
teristic information calculating section 437 for calculating 
subject characteristic information based on data in a spectral 

40 reflectance database 438 and giving this calculated informa- 
tion to the input profile preparing section 436. 

An input profile working section 434 prepares a device- 
independent color image by working an input profile that has 
bee selected from among the prepared input profiles 1 to N 

45 (4351 to 435N) by changing over a switch 432-1 onto the 
object photographic image. An image combining section 
439 combines device-independent color images by areas 
into one device-independent color image. A device value 
converting section 440 converts the combined devicc-inde- 

50 pendent color image into an output image that matches the 
device by referring to an output profile 4400. 

According to the present embodiment, a signal value of 
the color chart photographic image is checked. When this 
signal value is closer to the signal value of the object 

55 photographic image, the subject characteristic information is 
calculated based on the spectral reflectance of this color 
chart. As subject characteristic is calculated for each input 
image without having the subject characteristic information, 
it is possible to carry out a color reproduction in high 

60 precision. The subject characteristic information may be a 
correlation matrix or a base function that represents statis- 
tical characteristics of a spectral reflectance. 

Ninth Embodiment 

65 

A ninth embodiment of the present invention will be 
explained with reference to FIG. 22. Irie ninth embodiment 
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is characterized in that a color reproduction is carried out by 
using a program that achieves an illumination conversion 
processing by areas as one function of an image processing 
software for a digital camera. 

Referring to FIG. 22, an image file of an object obtained 
by photographing the object by a digital camera 600 is read. 
This image is displayed on a display screen 601, and a 
predetermined area of the image is designated with a mouse 
lo divide the area. Alternately, the area may be divided 
automatically by using RGB value rates. Thereafter, a user 
carries out a suitable illumination conversion for each area 
within the image by selecting information of subject char- 
acteristic information, observation illumination information, 
etc. An area designation and an illumination conversion are 
carried out repeatedly on the image to achieve illumination 
conversion by areas. After that, images in all areas are 
combined into one image, thereby to achieve color correc- 
tion in high precision. 

According to the above-described first to ninth embodi- ^ 0 
ments, an image is divided into images of a plurality of 
areas, and colors are converted suitably in each area. There- 
fore, it is possible to obtain a color reproduction system 
capable of carrying out color reproduction of all objects 
included in the image in high precision. 95 

Further, as objects within the photographed image are 
recognized correctly and a suitable color conversion is 
carried out for each object, it is possible lo obtain a color 
reproduction system capable of carrying out color reproduc- 
tion in high precision. 30 

Further, a color chart of a known spectral reflectance is 
photographed, and subject characteristic is calculated based 
on a spectral reflectance of a color chart closest to the signal 
value at the Lime of the photographing of an object. There- 
fore, it is possible to obtain a color reproduction system 35 
capable of carrying out color reproduction in high precision, 
without having subject characteristic information in 
advance. 



Tenth Embodiment 
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FIG. 24 shows a structure of a color reproduction system 
relating to a tenth embodiment of the present invention. This 
color reproduction system consists of a digital camera 801 as 
a color photographing section, a color correcting section 802 
having a color estimating function, and a CRT monitor 803. 

The digital camera 801 has three bands of red (R), green 
(G) and blue (B). The digital camera 801 photographs a 
subject, and outputs photographic image data (a subject 5Q 
photographic image) of three channels for each pixel, to the 
color correcting section 802. The photographic image data 
of each channel has a signal value of one byte (0 to 255). The 
color correcting section 802 converts the input photographic 
image data into a monitor input signal suitable for display in 55 
the CRT monitor 803, and outputs the converted signal to the 
CRT monitor 803. A subject image is displayed as a color 
image on the CRT monitor 803. 

The color correcting section 802 consists of a color 
converting unit 811, an TUT data memory unit 812, an LUT 60 
data calculating unit 813, and a characteristic data memory 
unit 814. 

Each unit of the color correcting section 802 will be 
explained in detail. First, the color converting unit 811 refers 
to data R', G' and B' in a lookup table (LUT) within the LUT 65 
data memory unit 812 by using RGB values as an address of 
the photographic image data that has been input from the 



digital camera 801. The color converting unit 811 then 
outputs the LUT data R', G ( and B 1 as monitor input signals 
to the CRT monitor 803. 

The LUT data memory unit 812 stores in advance LUT 
data calculated by the LUT data calculating unit 813. The 
LUT data consists of monitor input signals corresponding to 
all the signal values of the photographic image data, that is, 
R', G ? and B ! of 256 3 R\ G' and B' are one-byte data for each 
channel. 

The LUT data calculating unit 813 inputs data including 
a spectral sensitivity of the digital camera 801 (hereinafter to 
be referred lo as a digital camera spectral sensitivity), 
photographic illumination light spectrum, an observation 
illumination light spectrum, color matching functions, a 
subject spectral reflectance database, and a monitor profile 
from the characteristic data memory unit 814. The LUT data 
calculating unit 813 calculates the LUT data by using Lhcse 
data. Each data of the digital camera spectral sensitivity, the 
photographic illumination light spectrum, the observation 
illumination light spectrum, the color matching functions, 
the subject spectral reflectance database, and the monitor 
profile stored in the characteristic data memory unit 814 has 
the following construction. 

The data of the digital camera spectral sensitivity consists 
of sensitivity data of r (X), g (X) and b (X) each representing 
a spectral sensitivity of the digital camera 801 for colors R, 
G and B respectively that are set in 1 nm interval in a 
wavelength area of 380 nm to 780 nm. 

Each data of the photographic illumination light spectrum 
and the observation illumination light spectrum consists of 
light intensity data Era (X) and Lo (X) of the photographic 
illumination light spectrum and the observation illumination 
fight spectrum respectively set in 1 nm interval in a wave- 
length area of 380 nm to 780 nm. 

The data of the color matching functions are color match- 
ing functions x (X), y (X) and z (X) prescribed by the CIE. 
These data are interpolated lo have data in 1 nm interval in 
a wavelength area of 380 nm to 780 nm. 

The subject spectral reflectance database is a set of a large 
number of spectral reflectance data measured for objects in 
the real world. FIG. 25 shows an example of the subject 
spectral reflectance database. Each spectral reflectance data 
fi (X) (i=l to 1000) has spectral reflectance data in 1 nm 
interval in a wavelength area of 380 nm to 780 nm, and is 
managed by a registration number i. llie spectral reflectance 
data fi (X) registered in the subject spectral reflectance 
database includes data of objects photographed by the digital 
camera 801 such as, for example, a plant, a rock, a human 
skin, a color chart, a picture, a dye, a pigment, etc. The 
number of objects is assumed as 1000 in this embodiment. 
However, it is also possible to add data according to the 
need. The number of registration data is optional. 

The monitor profile consists of a matrix M and gradation 
correction data T. The matrix M is a matrix having elements 
of XrYrZr, XgYgzg and XbYbZb as XYZ tristimulus values 
of display colors in input signals (255, 0, 0), (0, 255, 0) and 
(0, 0, 255) of the CRT monitor 3. 



M = 



X r Xg Xt> 



The gradation data are table data yr (R) 7 yg (G) and yb (B) 
that give a relationship of output luminance to each input 
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The values of the photographic signals RiGiBi are stored 
in the database photographic signal memory unit 822 
together with the registration numbers i of the corresponding 
spectral reflectance data fi (K) as shown in FIG. 27. The 
sample data calculating unit 823 selects ten photographic 
signals RiGiBi in the small to large order of a distance Di 
from a predetermined sample photographic signal RsGsBs 
judged by the following expression. The sample data cal- 
culating unit 823 then outputs the registration numbers i and 
the sample photographic signal RsGsBs to the color con- 
version LUT calculating unit 824. 
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signal of R, G and B of the CRT monitor 803. Each data is 
normalized to have a maximum value of 1. 

FIG. 26 shows a structure of the TUT data calculating unit 
813. The LUT data calculating unit 813 consists of a 
database photographic signal calculating unit 821, a data- 
base photographic signal memory unit 822, a sample data 
calculating unit 823, and a color conversion LUT calculating 
unit 824, The database photographic signal calculating unit 
821 reads all data of the digital camera spectra] sensitivity, 
the photographic illumination light spectrum and the subject 
spectral reflectance database from the characteristic data 
memory unit 814 shown in FIG. 24, and calculates values of 
photographic signals RiGiBi for spectral reflectance data fi 
(X.) in the subject spectral reflectance database. The values of 
the photographic signals RiGiBi are estimated by the fol- 
lowing equations. 



a tristimulus value calculating unit 832, a monitor input 
signal calculating unit 833 and a data interpolating unit 834. 
The registration numbers i of the ten photographic signals 
RiGiBi selected by the sample data calculating unit 823 in 
FIG. 26 are input to the correlation matrix calculating unit 
831. The correlation matrix calculating unit 831 inputs the 
subject spectral reflectance data of the registration numbers 
i as subject spectral reflectance sample data from the char- 
acteristic data memory unit 814 shown in FIG. 24, and 
calculates a correlation matrix A, as shown by the following 
expression, as statistical data of the spectral reflectance of 
the subject. 



/3 I =y(/? 8 -R i ) :i +(G s -G i ) 2 +(^-B i ) 2 



FIG. 28 shows a concept of the sample photographic 
signal RsGsBs and a selection of the ten photographic 
signals RiGiBi nearest to the sample photographic signal. A 
mark of +shows the sample photographic signal RsGsBs, 
and black marks show the ten photographic signals RiGiBi 
nearest to the sample photographic signal. 

FIG. 29 shows a structure of the color conversion LUT 
calculating unit 824. The color conversion LUT calculating 
unit 824 consists of a correlation matrix calculating unit 831, 
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In the above expression, < > represents an average value 
of the ten spectral reflectance data. 
25 The correlation matrix A calculated by the correlation 
matrix calculating unit 831 is output to the tristimulus value 
calculating unit 832. The tristimulus value calculating unit 
832 calculates XYZ tristimulus values of the subject under 
the observation illumination light by the following expres- 
sion based on the correlation matrix A input from the 
correlation matrix calculating unit 831, the digital camera 
spectral sensitivity, the photographic illumination light spec- 
trum, the observation illumination light spectrum and the 
35 color matching functions input from the characteristic data 
memory unit 814 shown in FIG. 24, and the sample photo- 
graphic signal RsGsBs input from the sample data calculat- 
ing unit 823. 

40 C=K 1 K 2 " l O 
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The XYZ tristimulus values calculated by the tristimulus 
value calculating unit 832 are output to the monitor input 
signal calculating unit 833. The monitor input signal calcu- 
lating unit 833 converts the XYZ tristimulus values into a 
sample monitor input signal by using the monitor profile 
input from the characteristic data memory unit 814 shown in 
FIG. 24, and outputs the converted signal to the data 
interpolating unit 834. 

FIG. 30 shows a structure of the monitor input signal 
calculating unit 833. The monitor input signal calculating 
unit 833 consists of a matrix converting unit 841 and a 
gradation correcting unit 842, 

The matrix converting unit 841 converts the XYZ tris- 
timulus values input from the tristimulus value calculating 
unit 832 shown in FIG. 29 into R"sG"sB"s by the following 
expression based on the matrix M of the monitor profile 
input from the characteristic data memory unit 814 shown in 
FIG. 24. The matrix converting unit 841 outputs a result of 
the conversion to the gradation correcting unit 842. 

where 

P=(R/G/B/)' 

The gradation correcting unit 842 carries out a gradation 
correction of R"sG"sB"s input from the matrix converting 
unit 841 by the following expression based on an inverse 
function y _1 ( ) of gradation correction data y ( ) input from 
the characteristic data memory unit 814 shown in FIG. 24. 
The gradation correcting unit 842 outputs the gradation- 
corrected data as a sample monitor input signal R'sG'sB's to 
the data interpolating unit 834 shown in FIG. 29. 

The gradation correcting unit 842 carries out the above 
gradation correction processing to the predetermined sample 
photographic signals RsGsBs, and stores the results as 
sample monitor input signals R'sG'sB's in the data interpo- 
lating unit 834. The data interpolating unit 834 obtains by 
interpolation a monitor input signal corresponding to the 
value of the photographic signal from among the sample 
photographic signals RsGsBs, and obtains LUT data having 
monitor input signal values corresponding to all the photo- 
graphic signal values. 



FIG. 31 shows a structure of the data interpolating unit 
834. The data interpolating unit 834 consists of a data 
memory unit 851 and an interpolation calculating unit 852. 
The data memory unit 851 stores sample photographic 
signals input from the sample data calculating unit 823 
^ 5 shown in FIG. 26 and corresponding sample monitor input 
signal values input from the monitor input signal calculating 
unit 833 shown in FIG. 29. The interpolation calculating unit 
852 carries out an interpolation for all the photographic 
signal values based on the sample photographic signals near 
3Q the photographic signal and corresponding sample monitor 
input signals, thereby to obtain monitor input signals cor- 
responding to the photographic signals. The interpolation- 
calculating unit 852 then outputs a result of the interpolation 
to the LUT data memory unit 812 shown in FIG. 24. The 
35 method of interpolation to be used is optional. For signal 
values that cannot be interpolated based on the sample 
photographic signals and for signals that are not within the 
range from 0 to 255 T it is possible to cope with the situation 
by replacing signals by mapping to the near region. 
40 According to the color reproduction system having the 
above -described structure of the present embodiment, the 
color correcting section 802 estimates tristimulus color 
values of subjects by using statistical data A (the correlation 
matrix A in this example) of the spectral reflectance of the 
45 subjects. Colors are estimated by changing over the corre- 
lation matrix A as the statistical data according to photo- 
graphic signals of the subjects obtained by photographing 
the subjects by the digital camera 801. More specifically, the 
correlation matrix A is used by changing over the matrix 
50 according to the relationship between the spectral reflec- 
tance data fi (a,) for calculating the statistical data of the 
spectral reflectances and the subject photographic signals. 
With this arrangement, it is possible to estimate colors in 
higher precision than when the same statistical data is used 
55 for all the subject photographic signals. 

Eleventh Embodiment 

FIG. 32 shows a structure of a color reproduction system 
60 relating to an eleventh embodiment of the present invention. 
This color reproduction system consists of a digital camera 
801 as a color photographing section, a color correcting 
section 802', a CRT monitor 803, and a color chart 804. The 
color chart 804 consists of patches having 16 independent 
65 spectral reflectances (a plurality of objects having known 
spectral reflectances), each patch having a constant spectral 
reflectance. 
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The digital camera 801 has three bands of red (R), green 
(G) and blue (B). The digital camera 801 photographs a 
subject, and outputs photographic image data (a subject 
photographic image) of three channels for each pixel, to the 
color correcting section 802'. The photographic image data 5 
of each channel has a signal value of one byte (0 to 255). The 
color correcting section 802' converts the input photographic 
image data into a monitor input signal suitable for display in 
the CRT monitor 803, and outputs the converted signal to the 
CRT monitor 803. A subject image is displayed as a color 
image on the CRT monitor 803. 

The color correcting section 802' consists of a color 
converting unit 811, an LUT data memory unit 812, an LUT 
data calculating unit 815, and a characteristic data memory 
unit 816. "ITie color converting unit 811 and the LUT data 
memory unit 812 are similar to those explained in the tenth 15 
embodiment, and therefore, their explanation will be omitted 
here. 

The LUT data calculating unit 815 calculates LUT data 
based on color chart photographic signals obtained by 
photographing the color chart 804 by the digital camera 801, 20 
an observation illumination light spectrum, color matching 
functions, a subject spectral reflectance database, color chart 
spectral reflectance data, and a monitor profile that have 
been input from the characteristic data memory unit 816. 
The LUT data calculating unit 815 outputs the calculated 2 5 
LUT data to the LUT data memory unit 812. 

FIG. 33 shows a structure of the LUT data calculating unit 
815. The LUT data calculating unit 815 consists of a 
database photographic signal calculating unit 861, a data- 
base photographic signal memory unit 862, a sample data 3Q 
calculating unit 863, and color conversion LUT calculating 
unit 864. 

The database photographic signal calculating unit 861 
estimates photographic signals of each spectral reflectance 
of the subject spectral reflectance database, based on the ^ 
color chart photographic signals input from the digital 
camera 801 shown in FIG. 32 and the color chart spectral 
reflectance data input from the characteristic data memory 
unit 816. Within the database photographic signal calculat- 
ing unit 861, average signal value data (Rci, Gci, Bci) (i= 1 
to 16) of signals corresponding to 16 patches of the color 40 
chart 804 are first obtained from the color chart photo- 
graphic signals input from the digital camera 801. A pho- 
tographic signal (RiGiBi) of a spectral reflectance fi (k) in 
the subject spectral reflectance database is obtained from the 
following expression. 45 



( R> ^ 






- RcJt, > 


G; 














■ B C J6 ) 



V c;j6 ) 

In the above expression, cij is an expansion coefficient 
obtained by expanding an i-th spectral reflectance h* (X) of 55 
the subject spectral reflectance database based on a spectral 
reflectance pj (X) (j=l to 16) of each of the 16 patches of the 
color chart 804 as given by the following expression. 



i«5 

/■(A) = £ W (A) 



The photographic signals calculated by the database pho- 65 
tographic signal calculating unit 861 are stored in the 
database photographic signal memory unit 862. The data- 
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base photographic signal memory unit 862 and the sample 
data calculating unit 863 are similar to those explained in the 
tenth embodiment, and therefore, their explanation will be 
omitted here. 

FIG. 34 shows a structure of the color conversion LUT 
calculating unit 864. The color conversion LUT calculating 
unit 864 consists of a basis function calculating unit 871, a 
tristimulus value calculating unit 872, a monitor input signal 
calculating unit 873, and a data interpolating unit 874. 
Registration numbers i of ten photographic signal data 
selected by the sample data calculating unit 863 shown in 
FIG. 33 are input to the basis function calculating unit 871. 

The basis function calculating unit 871 receives an input 
of the subject spectral reflectance data of the registration 
numbers i as subject spectral reflectance sample data, from 
the characteristic data memory unit 816 shown in FIG. 32. 
The basis function calculating unit 871 then obtains basis 
functions pk (A,) (k=l to 3) of the input data as statistical data 
of the spectral reflectances of the subject. The basis func- 
tions pk (k) (k=l to 3) are obtained as cigen vectors of a 
correlation matrix of the spectral reflectances. 

The tristimulus value calculating unit 872 calculates XYZ 
tristimulus values of the subject under the observation 
illumination light as follows, based on the data of a sample 
signal RsGsBs input from the sample data calculating unit 
863 shown in FIG. 33, color chart photographic signals R (j), 
G (j) and B (j) (j=l to 16) input from the digital camera 801 
shown in FIG. 32, the basis functions pk (X) (k=l to 3) input 
from the basis function calculating unit 871, an observation 
illumination light spectrum Eo (A.), color matching functions 
x (X.), y (A) and z (X), and color chart spectral reflectances 
pj (A,) (j=l to 16) that have been input from the characteristic 
data memory unit 816 shown in FIG. 32. The tristimulus 
value calculating unit 872 then outputs the calculated XYZ 
tristimulus values of the subject to the monitor input signal 
calculating unit 873. 

In other words, the tristimulus value calculating unit 872 
calculates the spectral reflectances f (K) of the subject, and 
obtains the tristimulus values XYZ under the observation 
illumination light based on this calculation, from the fol- 
lowing expression. 

X= j x(A)£ 0 (A)/(A>/A 
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The spectral reflectances f (X) of the subject are obtained 
from the following expression. 

F=B(QD) *G 

where 

F=(f(380),f(381) 4 . . . ,f(780)) r 
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In the above expression, elements dk (j) (k=l to 3, j=l to 
16) of D are expansion functions obtained by expanding the 
basis functions based on the spectral reflectances of the 
patches of the color chart 804. 

16 

p k (\) = J^4 j) Pj a) (k = l to 3) 



The tristimulus values XYZ calculated by the tristimulus 
value calculating unit 872 are output to the monitor input 
signal calculating unit 873. The monitor input signal calcu- 
lating unit 873 and the data interpolating unit 874 are similar 
to those explained in the tenth embodiment, and therefore, 
their explanation will be omitted here. 

According to the color reproduction system having the 
above-described structure of the present embodiment, the 
color correcting section 802' estimates tristimulus color 
values of subjects under the observation illumination light, 
based on the spectral sensitivity and the color matching 
functions of the digital camera 801, the spectrum of the 
observation illumination light, and the statistical data of the 
spectral reflectances of the subjects (the basis functions pk 
(X) (k=l to 3) in the present example). Colors are estimated 
by changing over the basis functions pk (X.) (k=l to 3) as the 
statistical data according to photographic signals of the 
subjects. In other words, the basis functions pk (X) (k=l to 
3) are calculated based on the spectral reflectances of the 
subjects that give signal values close to the subject photo- 
graphic signals. With this arrangement, it becomes possible 
to carry out color estimation in high precision. 

Other Embodiments 

The present invention is not limited to the tenth and 
eleventh embodiments. It is also possible to implement the 
invention through various modifications as follows. 

(1) In the tenth and eleventh embodiments, the digital 
camera 801 of RGB three channels has been used as the 
color photographing section. However, the camera is not 
limited to this. It is also possible to use a digital camera of 
four or more channels. 

(2) In the tenth and eleventh embodiments, the CRT 
monitor 803 has been used as the color image output section. 
However, it is also possible to use other display unit such as 
a projector, or to use a hardcopy output unit Like a printer, 
instead of the CRT monitor. 

(3) In the tenth embodiment, the statistical data of the 
spectral reflectances of the subjects are calculated based on 
the spectral sensitivity of the camera and the photographic 
illumination light spectrum at the time of the photographing. 
However, it is also possible to calculate in advance the 
correlation matrix data of spectral sensitivity of a predeter- 
mined camera and photographic illumination light spectrum, 
and to use this calculated data. 

(4) In the eleventh embodiment, for photographing the 
subject and the color chart 804, it is possible to photograph 
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the subject and the color chart separately when the photo- 
graphing conditions are the same. Alternatively, it is also 
possible to photograph the subject and the color chart in one 
picture at the same time, and to separately take out the 
subject photographic signals and the color chart photo- 
graphic signals after the photographing. 

(5) In the eleventh embodiment, the number of patches 
included in the color chart 804 is optional. However, it is 
desirable that all the spectral reflectance data in the subject 
spectral reflectance database can be expanded in high pre- 
cision based on the spectral reflectances of the patches 
included in the color chart 804. 

As explained above, according to the color reproduction 
system of the tenth and eleventh embodiments, there is used 
statistical data of the spectral reflectances of subjects as 
15 foresight information based on the subject photographic 
signals obtained by photographing the subjects by the color 
photographing unit or the subject photographic signals and 
the object photographic signals obtained by photographing a 
plurality of objects of known spectral reflectances by using 
20 the color photographic unit. In estimating colors of the 
subjects under a predetermined illumination light, the sta- 
tistical data is used by changing over the data according to 
the subject photographic signals. With this arrangement, it is 
possible to estimate colors of the subjects in high precision 
based on a simple system structure using a color photo- 
graphing unit of a limited number of bands. As a result, it is 
possible to achieve an accurate color reproduction. 

Additional advantages and modifications will readily 
occur to those skilled in the art. Therefore, the invention in 
its broader aspects is not limited to the specific details and 
representative embodiments shown and described herein. 
Accordingly, various modifications may be made without 
departing from the spirit or scope of the general inventive 
concept as defined by the appended claims and their equiva- 
lents. 

What is claimed is: 

1. A color reproduction system comprising: 
a color image input section; and 

a color correcting section for correcting colors of an 
image of a subject obtained by photographing the 
subject by the color image input section, wherein 
the color correcting section comprises: 
an object recognizing section for dividing a subject image 
input from the color image input section into images of 
a plurality of areas, and selecting a suitable input profile 
45 from among a plurality of input profiles for each 

divided area; 

a device -independent color converting section for con- 
verting an image of each area into a device-independent 
color image by using the input profile selected from 
5q among the plurality of input profiles; 

an image combining section for combining the device- 
independent color images converted at divided areas 
into one device-independent color image; and 
a device value image converting section for converting 
55 the combined device-independent color image into an 

output device value image by using a predetermined 
output profile. 

2. A color reproduction system according to claim 1, 
wherein the input profile is prepared based on at least one of 
image input unit information that includes photographic 

60 characteristics of an image input unit used for the photo- 
graphing and information on a set state of the color image 
input section, observation illumination light information, 
and subject characteristic information that represents statis- 
tical characteristics of spectral reflectances of the photo- 

65 graphed subject. 



